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ABSTRACT 

The purpose of the 6-year Minnesota Student 
Characteristics and Occupationally Related Education Project is to 
identify criteria which are useful to counselors and others in the 
selection and counseling of post-high school vocational-technical 
students as they choose specific vocational-technical curriculums- 
The initial phase collected preenrollment data on 17,500 Minnesota 
area vocational-technical school applicants. This phase is a pilot 
followup of 1,128 graduates of Minnesota Area Vocational-Technical 
Schools 1 year after graduation, to gather data related to their 
relative success on the job. The specific purpose is to test the 
effectiveness of various incentives designed to maximize the 
percentage of mailed guestionnaire returns. Data were based on 576 
(52 percent) responses obtained during a 4-week period. Other 
variables included time interval, sex, and occupational cluster. 
Percentage of returns can be increased by introducing incentives into 
the followup procedures, A related document is available as ED 025 
658, (CH) 
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QUESTIONNAIRE FOLLOW-UP RETURNS AS A 
FUNCTION OF INCENTIVES AND RESPONDER, CHARACTERISTICS 



BACKGROUND 

Project MINI-SCORE (Minnesota Student Characteristics and Occupationally 
Related Education) is a comprehensivej, six-year research project designed to assess 
the basic competencies and characteristics of vocational students to determine the 
degree to which each is directly associated with the successful performance of 
those who graduate. The initial phase of the study gathered pre-enrollment test 
data on approximately 17»500 applicants to the post high school area vocational- 
technical schools of Minnesota during the period from September 1, 1966 to Octo- 
ber If 1968, Some of these students were accepted to vocational programs and 
graduated. The present phase involves a follow-up of students one year after 
graduation to gather data related to their relative success on the job. The next 
phase will be concerned with determining which of the variables measured prior to 
enrollment are useful in predicting success on the job, 

STATEMENT OF THE PROBLEM 

By the summer of 1968s the first group of 1128 graduates had been graduated 
for one year. Because of the large scale of the total follow-up, it was decided 
to use this initial group to test various procedures to maximize the percentage 
of mailed questionnaire returns. It was also decided to analyze the response 
patterns to gain information regarding the characteristics of respondents and non- 
respondents, with the particular aim of identifying, as early as possible, poten- 
tially "reluctant”' groups which would require special follow-up strategies. 
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REVIEW OF THE LITERATURE 

Considerable attention has been given to increasing the rate of return of 
questionnaires through variations in construction and follow-up procedures. Scott 
(1961} in an extensive study for the British Government Social Survey, investi- 
gated the effect of: (a) stamped versus business reply envelopes, (b) official 
sponsorship, (c) personalizing the cover letter, (d) a letterhead, (e) colored 
paper, and a number of other factorso He found -tamped envelopes and official 
sponsorship to be effective. Reporting on response to Project TALENT question- 
naires, Orr and Neyman (1965) found the average cost for handling and mailing 
the questionnaire to be 58o6 cents per respondent using a total of four mailings 
and achieving some 69.7 per cent response from an original 88,000 subjects. They 
also found that the length of the questionnaire affected the return rate, with 
approximately 37 per cent response for a four page questionnaire versus approxi- 
mately 30 per cent for an eight page questionnaire 35 days after first mailingo 
They further reported that the peak return rate occurred 12 days after mailing 
and that response rate was positively correlated with aptitude scores o 

Moore (1941) found that a typewritten letter of transmittal increased the 
return rate significantly over a duplicated letter. Others have suggested that 
neatness rather than the actual method of reproduction is the important factor. 
Sletto (1940) found that the nature of the appeal for assistance made in the 
cover letter affected the rate of return, with the most effective for his group 
of former college students being an appeal to help improve education for others® 
Gerberich and Mason (1948) found that whether or not a subject was asked to sign 
the questionnaire made little difference in the item response® Others have 
pointed out that the nature of the material being investigated would possibly 
affect the return rate of confidential versus non"Confidential questionnaires® 
Listed below in summary form are a number of techniques or variables that 
persons have either shown or have hypothesized might affect the rate of return 



of questionnaires.. The list is categorized into: (a) questionnaire construction 

and (b) follow-up procedures » 

Questionnaire Constructlo^n 

Some evidence and/or opinion suggests that returns may be increased by con- 
structing a questionnaire that: 



1. is logical in question organization. ^ . u • 

2. is clear and unambigurous in wording ■■■ unbiased in phrasing. 

3. is non-repretitive and non-trivial. 

4. is as brief as possible. 

5. is attractively reproduced. 

6» avoids the use of the word "questionnaire • 

7. keeps directions brief, clear, and distinct, 

8o is printed on colored paper. 



Follow-up Procedures 

Some evidence and/or opinion suggests that returns may be increased by using 

follow-up techniques that: 

1. include a return addressed stamped envelope, 

2o use a stamped rather than a business reply envelope. . j.* i 

3o include official sponsorship by a party respected by the potential 

respondent. 

4o include a personalized accompanying letter, 

5, consider the time of mailing the questionnaire (day of the week and 
time of the year). 

6, include assurance of confidentiality. 

7, offer a summary of results. 

8, contain a deadline date for returning. 



OBJECTIVES 



The questions examined in the study were: 

1, Are there differences in total questionnaire returns among groups who 
receive different incentives to encourage them to return the question- 
naires? 

2. Are there differences in total questionnaire returns among groups who 
receive different numbers of incentives to encourage them to return the 

questionnaires? 

3o Are there differences between groups that receive different numbers of, 
Incentives in terms of the amount of time it takes group members to 
return their questionnaires? 
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4» Does the sex of the subject affect the probability of responding and is 
the effect of the different numbers of incentives different for males 
and females? 

5o Does the occupational skill level of the subject affect the probability 
of responding and is the effect of different numbers of incentives dif- 
ferent for different occupational skill levels? 



PROCEDURE 



Population 

The population consisted of 1128 graduates from the twenty-four^ post-high 
school, Minnesota Area Vocational-Technical Schools taking part in Project MINI- 
SCORE who had graduated 12 months prior to the study for whom data had been 
reported-. The study was undertaken during the Winter of 1967. The graduates 
represented 26 different trade areas ranging from practical nursing to clerical 
training and from electronics to weldingt^ These graduates were the first of the 
Project MINI-SCORE students available for follow-up. One possible limitation 
should be noted; because they were the first graduates, it is probable that the 
group represented the shorter-term vocational programs of the total vocational 
offerings. 



Method 

The names of 1100 graduates were randomly assigned to 11 groups of 100 per- 
sons each. The remaining 28 subjects were used to informally pilot test the ques- 
tionnaire two weeks before the main follow-up study began. Treatments were ran- 
domly assigned to groups. 

After persons were assigned to groups and groups assigned to treatments, the 
mailings were prepared. All 1100 questionnaires were mailed at the same time. A 
daily record was kept of returned questionnaires. No reminders or other communi- 
cations were sent during the four week period of the study. 
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Treatments 



The treatments for the 11 groups were as follows. All treatments included a 
stamped return addressed envelope. 



1. Control treatment ~ zero incentives - regular questionnaire and cover 
letter. The regular questionnaire consisted of sections on additional 
training since graduation, employment information including a work, his- 
tory, and job satisfaction. The questionnaire was printed on plain white 
paper and was lightly overprinted in red with the word CONFIDENTIA.L. The 
personalized cover letter briefly explained the intent of the follow-up 
and how important it was to the success of the project that the question- 
naire be completed and returned. 

2. Pencil treatment - one incentive - A short pre-sharpened *'golf" pencil 
inscribed with the phrase, THANKS/UNIVERSITY OF MINNESOTA, was inc.luded 
with the regular questionnaire. The cover letter was essentially the 
same as the standard cover letter except for a brief reference to the 
enclosed pencil. 

3. Coffee treatment - one incentive - A "cup size" packet of instant coffee 
and the regular cover letter altered slightly to note the enclosed coffee 
were included with the regular questionnaire. 

4. Colored questionnaire treatment - one incentive -• This treatment was 
identical to the control treatment, except the questionnaire itself was 
printed on light green paper. 



5. Pre-letter treatment - one incentive - The questionnaire and cover letter 
were identical to the control treatment. However, one week prior to 
mailing the questionnaire, a letter was sent to each person in the group 
indicating that they would be asked to help with the study and that a 
questionnaire would soon be sent to them. 

6. Pencil and colored questionnaire treatment - two incentives - This treat- 
ment combined treatments 2 and 4. 

7. Pencil and pre- letter treatment - two incentives - This treatment com- 
bined treatments 2 and 5. 



8. Pencil, colored questionnaire and pre- letter treatment - three incen- 
tives - This treatment combined treatments 2, 4 and 5. 

9» Coffee and colored questionnaire treatment - two incentives - This treat- 
ment combined treatments 3 and 4. 

10. Coffee and pre- letter treatment - two incentives - This treatment com- 
bined treatments 3 and 5. 

11. Coffee, colored questionnaire and pre- letter treatment - three incen- 
tives - This treatment combined treatments 3, 4 and 5. 














RESULTS 



Bv treatment 
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The total number of returns for eaoh group at the end of the four week study 
period is reported in Tabie I. The percentage of respondents is also reported. 



TABLE I 



OVERALL RESPONSE PER TREATMENT GROUP 



Group 1 (control) 

Group 2 (pencil) 

Group 3 (coffee) 

Group 4 (color) 

Group 5 (pre-lei,ter) 

Group 6 (pencil and color) 

Group 7 (pencil and pre- letter) 

Group 8 (pencil? color and pre- letter) 
Group 9 (coffee and color) 

Group 10 (coffee and pre- letter) 

Group 11 (coffee, color and pre-letter 



Actual'^ Number Percentage 



Group Size Returned RgtuijQed 



1 98 


42 


42o9 


95 


52 


54u7 


97 


48 


49.5 


98 


49 


50.0 


97 


50 


51.5 


97 


51 


52.6 


97 


56 


57.7 


96 


60 


62.5 


99 


55 


55.6 


98 


53 


54ol 


95 


60 


63.2 



A chi-square test of the differences in return rates between the eleven 
groups yielded a chi-square value of 13.28 (.20ZP^.30). It is interesting to 
note that although the probability ieA,el is not impressive, the differences were 
in the expected direction and that the range between the response rate of the con- 
troi group (the lowest) and the coffee, colored questionnaire and pre-letter group 

(the highest) was 20o3 per cent. 



Bv Number of Incentives 

The eleven treatments were clustered on the basis of the number of incentive 



^he actual group sizes used in the calculations were arrived at by subtracting 
the number of questionnaires per group undelivered because of insufficient address 
frL the original 100 per group. The problem was to study the effectiveness of 
the incentives. Therefore, if the questionnaire did not reach an individual the 
incentives had no possibility of being effective. 
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in each treatment 3 ranging from zero incentives for the control treatment to three 
incentives for treatments 8 and 11 o The actual number and the percentage of per- 
sons returning the questionnaires for each number of incentives is shown in Table 

JL o 



TABLE II 

RETURN PER NUMBER OF INCENTIVES 
Number of Incentives 



Returned 
Non- Returned 



Total 



0 1 2 3 Total 



42(42o9^) 1 


199(51.4^) 


1 215(55o0^) 


120(62o8^) 


j576(54.0^) j 


|^57ol^) 

D , 1 


[188(4806^) 


176(45.0^) 


7l(37o2^) 1 


491(46.0^) j 


98(100^) 


[387(100^) 


1 391 (.100^) 1 


191(100^) 


1067 



2 

A chi-square test of the data reported in Table II yielded a X value of 
12o09 (P-tdjfOl)® The differences in response rate consistently and significantly 
favored an increased number of incentives. 



Incentives by Time Interval 

To study the effect of the number of incentives on the rate of responsog, the 
four week study period was divided into 8 intervals 3 with the interval periods 
ending on Tuesdays and Fridays. The number of questionnaires returned during each 
interval was recorded for each of the four incentive clusters. Although the 

•I 

intervals were unequal in length - 4 days and 3 days - the longer period included 
the weekend. 

The per cent of returns per time interval for each incentive cluster is pre- 
sented in Table III. The total number of returns for each incentive cluster 
equals 100 per cent. 

The responses began in the third time interval j were maximum at the fourth 
time interval (equivalent to the end of the second week), and tapered off rapidly 



TABLE III 



PER CENT OF RETURNS DURING EACH TIME PERIOD 
FOR DIFFERENT NUMBER OF INCENTIVES 



Number 



Incen“ 

tives 



by the eighth intervale The superiority of increased numbers of incentives in 
interval 3 and especially in interval 4 suggests that incentives might be effec- 
tive in securing a more rapid returno By the end of the fourth interval, 51.7 
per cent of the total returns of the three incentive cluster were in, but only 
33.3 per cent of the returns of the zero incentive cluster. However, a chi-square 
test of the independence of the number of incentives and the return time indicated 
the differences in response rates were not highly significant (X^=16.13, .20^P^.30)o 

Sex 

The probability of a questionnaire being returned was found to be signifi- 
cantly related to the sex of the subject. With an overall return rate (at the 
end of the four week study period) of 54.0 per centg 60.2 per cent of the females 
had responded versus 41o8 per cent of the males. Actual numbers are shown in 
Table IV below. 

A chi-square test of the difference in the return rates of males and females 
yielded an X" value of 32.45 (P^oOOl). For this population, females were more 
likely to return questionnaires than were males. 



Time I rtterval 



MW — ■■■! 

1 


2 


3 


— r. 

i 


5 


6 


7 1 


8 


Total 




lA* lat* 


0 ] 
(OoO^) 


r™T4“"”“' 

1 (33o3^) 


16 

(38.15^) 


6 

(14,3%) 


5 i 

(11.9?^) 


1 

(2.4%) 


42 

(100^0^1. 




t'X) 1 


1 

(5.0^) 


79 

(39.7g^) 


55 1 

(27s.6Ml_ 


36 

J. 18 oJL^L 


14 

llml 


5 

(2.5^) 


199 

(10P..Q%1- 






15 

1 (7.0^) 


78 ] 

(36.3^) 


I 62 
(28,8%) 


35 

(16.3^) 


13 1 

(6o0^) 


r"i2 1 

(5o6%) 


215 

(100.0%) 




— 


1 ^ 

(5.0^) 


56 

(46.7%) 


31 

J.25.81L 


19 

(15.8^) 


7 

(5.8%) 


1 

(0.8%) 


120 

UloOiML, 



TABLE IV 



NUMBERS AND PERCENTAGE OF QUESTIONNAIRE 
RETURJ^S FOE MALES AND FEMALES 



Rexurnsd 
Non- Returned 
Total 



Male Female Overall 



151 (41,8^) 


425 (60,2^) 


576 (54o0^) 


210 (58o2^) 


281 (39o8?^) 


491 (46.0?^) 


361 (lOOoO^) 


706 ilOQ.0%) 


1067 (100,0?^) 



The data reported in Tables V and VI suggest that the number of incentives 
is more important for males than for females. The male response rate rose from 
27o6 per cent for zero incentives to 57,6 per cent for three incentives, a dif- 



ference of 30 per cent. For females, the percentages were 49,3 and 65,6; a dif 
ference of only 16o3 per cent. 



TABLE V 

RETURNS FOR MALES RECEIVING 
DIFFERENT NUMBERS OF INCENTIVES 



Return 

Non-Return 



0 1 2 3 Total 



8 

(27,6^) 


51 

l37~8%) 


54 

(41.25«) 


38 

(57.6^) 


151 

.■(41.8a__ 


21 


84 


77 


28 

_.C42.4^) J 


210 

(58,2%) 



(100,0^) (100,05^) (lOOoO^) (100,03^) (100,03^) 



TABLE VI 

RETURNS FOR FFiAALES RECEIVING 
DIFFEREJ^T NUMBERS OF INCENTIVES 





0 


1 


2 


3 


Total 


Returned 


34 ^ 

(49..3^) 


148 

(58,7^) 


161 1 
(61.,«) 


1 ' 82 
(65.6%) 


425 

(60.2%) 


Non-Returned 


35 

(50,7%) 


104 

(41.3^) 


99 

.__138.1^ 


43 

(34,4%) 


281 

(39.8%) 




69 

(100,0%) 


252 

(100„0%) 


260 

(100,0%) 


125 

(xoo,o%) 


706 

(100,0%) 
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A chi-$quar© t©st of th© diff@r@nc©s in return ratts for groups of mal«s 
receiving different numbers of .Incentives yielded a X value of 10«07 (P^*Q5). 

The same test for females yielded a value of 5-«52 (•10i^P<l«20) . For both males 
and females., as the number of incentives increased the return rate also increased* 

1 

Occupational Cluster 

An attempt was made to determine the effect of the occupational skill level 
of a person's training on the likelihood of responding. In order to do this, each 
curriculum area or occupation represented in the study was assigned to an occupa- 
tional group according to the first number of the DOT (Dictionary of Occupational 
Titles) classification for the given occupation. This classification assigns 
occupations to occupat.ional groups according to information pertaining to work 
field, purpose, material, product, subject matter, service, generic term, and/or 

industry. 

After DOT numbers were assigned to each of the 26 trade areas, they were 
grouped into 3 clusters. The clusters were assigned tne letters? A for DOT 0,1 
(Professional, Technical and Managerial); B for DOT 2 (Clerical and Sales); and 
C for DOT groups 3 through 9 (Service, Skilled, and Miscellaneous Occupations). 
The actual number and percentage of responses for each group is shown in Table 

VI I o 



A 

Occupational g 
Cluster n 



•^The information related to giouping the occupations was taken from the Die 
tionary of Occupational Titles, Vo.lumej 1 and 2, 1965 edition. 



TAB.LE VII 

NUMBER AND PERCENTAGE OF RESPONSES 
FROM THE THREE OCCUPATIONA.L CLUSTERS 



Resoonse Perc entage Return 



222 of 402 


55%‘ 


256 of 431 


59?^ 


98 of 234 


42^ 




• 11 * 



A chi-iquar© test of th© data reported in Table VII yielded a valu© of 19*13 
(P 4^.001)* It appears that there are significant differences in the response 
rates of persons trained at different occupational skill levels. Persons in the 
clerical and sales cluster responded most frequently while persons in the skilled 
cluster responded least frequently. 

Tables VIII, IX, and X summarize data representing the differences in the 
response rates of groups of persons at each of the three occupational levels who 
received different numbers of incentives. 



TABLE VIII 

RETURNS FOR PERSONS IN CLUSTER-A 
RECEIVING DIFFERENT NUMBERS OF INCENTIVES 



Number of Incentives 



0 



Total 





18 


79 


83 


42 


222 


Returned 


(47.4^) 


(53.4^) 


(54^® 1 


(66.7%) 


(55.2%) 


Non-Returned 


20 

(52., 6%) 


69 

(46.6^) 


70 

(45.8%) 


21 

(33.3^U 


1 180 
i (44.8^) 




38 


148 


153 


63 


402 




{100. d^) 


(100.0%) 


(100.0%) 


(100.0%) 


(100.0%) 



TABLE IX 

RETURNS FOR PERSONS IN CLUSTER-B 
RECEIVING DIFFERENT NUMBERS OF INCENTIVES 

Number of Incentives 



Returned 
Non “Returned 



0 1 2 3 Total 



17 

(47.25^) 


85 

(56.35«L_ 


98 

_160,9^) 


56 

(67.5%) 


256 

(59.4^) 


19 1 

(52o8%) 


66 

(43.7%) 


63 

(39.1ML_ 


27 

(32o5%) 


175 

(40.6^) 



36 151 161 83 431 

(lOO.w/,) (100. («) (100.0^) (100. 0!«) (100. 0!«) 
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TABLE X 

RETURNS FOR PERSONS IN CLUSTER-C 
RECEIVING DIFFERENT NUMBERS OF INCENTIVES 



Number of Incentives 





0 


1 


2 


3 


Total 


Returned 


' 7 

(29.2^) 


35 

(39. 8«) 


34 

(44.2^) 


22 

(48.9^) 


98 

(41.9^) 


Non- Re turned 


17 

(70.8^) 


53 

(60.2^)._ 


43 

(55n8^) 


23 

(5K.l^ 


136 

■ (58.1^) 



24 88 77 45 234 

( 100 . 0 ^) ( 100 . 0 ^) ( 100 . 0 ?^) ( 100 . 0 ^) ( 100 . 0 ?^) 



Although the differences in response rates among groups receiving different 

2 

numbers of incentives in each cluster were not highly significant (cluster A-X — 
4»53-.20^:;^P^»30j cluster B-X^ = 5<,21“ol0^P^..20, cluster C-X^ = 2.83-.30.*dP^.50) j 
in all three cases an increase in the number of incentives was accompanied by an 
increase in returns. 



SUMMARY AND CONCLUSIONS 

The population of 1100 post-high school vocational school graduates was ran- 
domly divided into eleven groups of 100 each to receive a follow-up questionnaire. 
Each of the eleven groups received a different follow-up treatment. It was found 
that the overall percentage of returns varied from a low of 42.9 per cent for the 
group with no special treatment to a high of 63o2 per cent for one group with 
three different incentives. The response rates elicited to the four different 
numbers of incentives (incentive clusters zero - three) were found to be differ- 
ent with a X^ of 12.09 (P.<.01). 

Time interval analysis showed the greatest response for all groups to be 
near the end of the second week after the questionnaire was sent, which is simi- 



o 



lar to what Orr and Neyman (1965) found relative to Project TALENT questionnaire 
returns. The data indicated that as the number of incentives increased, there 
was a slight decrease in the length of response time. 



■m 
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In terms of sex, females responded much better overall than males — 60,2 
per cent for females, 41.8 per cent for males (X^ = 32,45 j P^.OOl). Interest- 
ingly, although the number of incentives seemed important in stimulating responses 
from both sexes, it appeared much more important for males than for females. Of 
the males, only 27.6 per cent in the zero incentive category returned their ques- 
tionnaires, while 57.6 per cent of those in the three incentive category returned 
their questionnaires (a range of 30o0 per cent). For the females, the figures 
were 49,3 per cent in the zero incentive category and 65,6 per cent in the three 

incentive category (a range of 16.3 per cent). 

An attempt was made to determine response rate differences among occupational 
clusters. As the skill level increased, the number of returns tended to increase 
(X^ = 19.13, P^l.OOl). This finding tends to support that of Orr and Neyman who 
reported that response rate is positively correlated with aptitude. Also, there 
is a tendency for the response rate to increase as the number of incentives 
increases for all three occupational clusters o 

There is little doubt that follow-up returns might be increased substantially 
by increasing the number of incentives and, therefore, the novelty of the follow- 
up procedures. 

POSTSCRIPT 

After the four week period of the questionnaire study, additional follow-up 
procedures were undertaken. All eleven treatment groups received the same second 
follow-up procedure consisting of the mailing of another cover letter and a green 
colored questionnaire. By eight weeks after the beginning of the study, the total 
response was 76.1 per cent. It is interesting to note that the proportion of pre- 
viously unreturned questionnaires which were returned after the second follow-up 
continued to favor those treatments with greater numbers of incentives, as shown 
in Table XI. 
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TABLE XI 

PERCENTAGE OF PREVIOUSLY UNRETURNED QUESTIONNAIRES 
RETURNED AFTER THE SECOND FOLLOW-UP 



Original Groups (Number of Incentives) 



0 


1 


2 


3 


43^ 


49 % 




49 % 


53^ 



At this pointj addressess of the non-respondents were checked through such 
sources as the Minnesota Drivers License Bureau, the Minnesota Drivers License 
Plates Bureau, and local telephone directories. Fourteen weeks after the begin- 
ning of the follow-up, the overall response rate was approximately 82 per cent* 
The response rate for males was 75*3 per cent and the response rate for females 
was 84,8 per cent. 



o 
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